Papers with entity typing
Efficient Entity Embedding Construction from Type Knowledge for BERT (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing work has shown advantages of incorporating knowledge graphs (KGs) into BERT for various NLP tasks. |
| Approach: | They propose to integrate knowledge graphs into BERT to train entity embeddings to include rich information of factual knowledge. |
| Outcome: | The proposed models perform very well when combined with context. |
Knowledge Enhanced Contextual Word Representations (D19-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, Noah A. Smith
| Challenge: | Existing methods to embed knowledge bases into large pre-training models do not contain any explicit grounding to real world entities and are difficult to recover factual knowledge. |
| Approach: | They propose a method to embed multiple knowledge bases (KBs) into large pretrained models with a Knowledge Attention and Recontextualization mechanism. |
| Outcome: | The proposed model improves perplexity, ability to recall facts and word sense disambiguation. |
CogIE: An Information Extraction Toolkit for Bridging Texts and CogNet (2021.acl-demo)
Copied to clipboard
| Challenge: | CogNet is a knowledge base that integrates three types of knowledge: linguistic knowledge, world knowledge and commonsense knowledge. |
| Approach: | They propose an information extraction toolkit called CogIE that is a bridge connecting raw texts and CogNet. |
| Outcome: | The proposed toolkit can ground raw texts to CogNet and leverage different types of knowledge to enrich extracted results. |
Comprehensive Multi-Dataset Evaluation of Reading Comprehension (D19-58)
Copied to clipboard
| Challenge: | Recent research aims to facilitate training and evaluation on several reading comprehension datasets at the same time. |
| Approach: | They propose an evaluation server that reports performance on seven diverse reading comprehension datasets and includes synthetic augmentations to test models' ability to handle out-of-domain questions. |
| Outcome: | The evaluation server performs on seven reading comprehension datasets, and collects and includes synthetic augmentations for these datasets to test models' ability to handle out-of-domain questions. |
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2021.findings-acl)
Copied to clipboard
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, Ming Zhou
| Challenge: | Existing methods for injecting knowledge into pre-trained models are inconsistent and can flush out knowledge when multiple kinds of knowledge are injected. |
| Approach: | They propose a framework that retains the original parameters of pre-trained models fixed and supports the development of versatile knowledge-infused models. |
| Outcome: | The proposed framework retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused models. |
Unified Semantic Typing with Meaningful Label Inference (2022.naacl-main)
Copied to clipboard
| Challenge: | Semantic typing aims at classifying tokens into semantic categories such as relations, entity types, and event types. |
| Approach: | They propose a unified framework for semantic typing that captures label semantics by projecting both inputs and labels into a joint semantic embedding space. |
| Outcome: | The proposed framework achieves strong performance across three semantic typing tasks. |
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)
Copied to clipboard
Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou
| Challenge: | Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering. |
| Approach: | They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text. |
| Outcome: | The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings. |
Conditional set generation using Seq2seq models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Several NLP tasks are instances of set generation. |
| Approach: | They propose a model-independent data augmentation approach that enlarges the model with the signals of order-invariance and cardinality. |
| Outcome: | The proposed method improves performance on four benchmark datasets with no additional annotations. |
Multi-Multi-View Learning: Multilingual and Multi-Representation Entity Typing (D18-1)
Copied to clipboard
| Challenge: | Accurate and complete knowledge bases (KBs) are paramount in NLP. |
| Approach: | They employ multiview learning for increasing the accuracy and coverage of entity type information in KBs by taking high- and low-resource languages from Wikipedia. |
| Outcome: | The proposed learning improves the accuracy and coverage of knowledge bases (KBs) by combining language and representation. |
KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained Relationships (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pretrained language models focus on entity information and ignore fine-grained relationships between entities. |
| Approach: | They propose to incorporate KG into the language learning process to obtain a KG-enhanced pretrained Language Model. |
| Outcome: | The proposed model improves on several knowledge-driven tasks, such as entity typing and relation classification, compared with the state-of-the-art knowledge-enhanced PLMs. |
Syntax-Enhanced Pre-trained Model (2021.acl-long)
Copied to clipboard
Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan
| Challenge: | Existing methods that use syntax of text in pre-training and fine-tuning suffer from discrepancy between the two stages. |
| Approach: | They propose a model that utilizes the syntactic structure of text in pre-training and fine-tuning stages. |
| Outcome: | The proposed model achieves state-of-the-art on six public benchmark datasets. |
Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to recognize named entities have been criticized for their performance on flat NER but fail to handle nested entities. |
| Approach: | They propose to use a span-based constituency parser to tackle nested NER . they use lexicalized constituency trees to model nesting entities . |
| Outcome: | The proposed method achieves state-of-the-art performance on ACE2004, ACE2005 and NNE, and competitive performance on the GENIA platform. |
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained transformer models have shown great success in improving performance on downstream tasks, but fine-tuning on a new task still requires large amounts of labeled data. |
| Approach: | They propose a method which allows optimization-based meta-learning across tasks . they use transformers to train transformer models and find better generalizations . |
| Outcome: | The proposed method outperforms self-supervised training and pre-trained models on 17 NLP tasks. |
Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field (2022.emnlp-main)
Copied to clipboard
| Challenge: | Entity typing assigns semantic types to entities mentioned in text. |
| Approach: | They propose to use an undirected graphical model to formulate the UFET problem by combining unary potentials with a pairwise conditional random field model. |
| Outcome: | The proposed model outperforms the existing model with little cost and is thousands of times faster than the existing neural network module. |
Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing entity typing models are subject to spurious correlations due to shortcuts and biased training. |
| Approach: | They propose a method to augment existing model biases by combining spurious correlations with debiasedcounterparts to improve generalization. |
| Outcome: | The proposed method improves generalization of different entity typing models on the original and debiased test sets. |
Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art (SOTA) methods use the cross-encoder architecture to concatenate a mention (and its context) with each type and feed it into a pretrained language model (PLM) to score their relevance. |
| Approach: | They propose to perform entity typing in a recall-expand-filter manner and use a novel model to encode and score all these K candidates in one forward pass. |
| Outcome: | The proposed method is thousands of times faster than the CE-based architecture and is very efficient in fine-grained (130 types) and coarse-grain (9 types) entity typing. |